The present invention relates to the detection of moving objects in a video image sequence, in particular a method comprising the steps:
a) determining distinctive feature points in one image of a pair of consecutive images of the video image sequence,
b) determining a mathematical transformation for imaging one of the two images of the pair of images onto the other of the two images of the pair of images, using the distinctive feature points determined in step a),
c) determining a difference image as a difference between the images of the pair of images transformed onto one another by means of transformation determined in step b),
d) determining distinctive image variation points in the difference image determined in step c),
e) determining object points from the distinctive image variation points determined in step d).
Methods of this type are known, for example, from the prior art for the detection of moving objects in camera image sequences of a pan and tilt camera installed on board an aircraft. The detection is carried out with the aid of a so-called change detection, in which chronologically consecutive video images of the camera image sequence are compared to one another and image variations are determined. The moving object or objects recorded in the scenery can then be determined in real time from the image variations determined.
With respect to the reliability or quality of the known detection methods, the following problems in particular result:
If, in the course of the respective video image sequence, the recorded “image section” is changed per se, as is the case, for example with a moving and/or panning and/or tilting camera due to the changes of viewing angle and/or observation distance associated therewith (e.g., with the use of an image sensor device on board a vehicle, in particular an aircraft), the image variations resulting solely therefrom first must be compensated, which is also referred to as “image registration.” With this image registration, however, substantial inaccuracies can occur in the known methods. This problem relates above all to above-referenced steps a) through c).
Moreover, chronological changes to the recording conditions (e.g., illumination) as well as the not completely avoidable recording inaccuracies that occur in practice (e.g., due to image recording noise, contrast compensation etc.) can impair the reliability or quality of the object detection method. This can cause “false alarms,” i.e., incorrect image detections.
Exemplary embodiments of the present invention provide an objection detection technique of the type mentioned at the outset that is as robust as possible with respect to disturbing influences and provides a low false alarm rate.
According to a first aspect of the invention, step d), that is, the determination of “distinctive image variation points” in the previously detected difference image comprises the following steps:
d1) establishing an image variation threshold value and determining image variation points as those points in the difference image determined in step c) the absolute image brightness value of which exceeds the image variation threshold value,
d2) analyzing the quality of the image variation points determined in step d1) based on at least one predetermined quality criterion,
d3) if the quality criterion is met, establishing the image variation points determined in step d1) as the distinctive image variation points determined in step d) otherwise repetition of steps d1) and d2) with an image variation threshold value established in a changed manner.
The basic concept of this method design lies in that during the determination of distinctive image variation points in the previously determined difference image, an “adaptive” image variation threshold value is provided, which although initially is established in some manner in order to determine image variation points on this “experimental basis,” this initial establishment is then revised as needed and a new determination of the image variation points is repeated on the basis of a image variation threshold value established in a changed manner (if a predetermined criterion has not been met). One or optionally more of such repetitions of steps d1) and d2) with the aim of meeting (or at least “better meeting”) the quality criterion advantageously render possible an improvement in the robustness and a reduction in the false alarm rate of the object detection.
According to an embodiment, for which the applicant reserves the right to claim independent protection (in the sense of an independent second inventive aspect), it is provided that step a) comprises the following steps:
a1) detecting feature points in one of the two images of the pair of images, respectively in the vicinity of grid points of a grid established in advance and laid over this image,
a2) determining the distinctive feature points as a selection of the feature points detected in step a1) using at least one predetermined selection criterion.
According to one embodiment, for which the applicant reserves the right to claim impendent protection (in the sense of an independent third inventive aspect), it is provided that step b) comprises the following steps:
b1) determining displacement vectors for each of the distinctive feature points determined in step a) by determining the corresponding distinctive feature points in the other of the two images of the pair of images,
b2) determining displacement vectors to be used further as a selection of the displacement vectors determined in step b1), based on at least one predetermined selection criterion,
b3) calculating the mathematical transformation on the basis of the displacement vectors selected in step b2).
In one embodiment step c) comprises the following steps:
c1) applying the mathematical transformation for transforming onto one another the two images of the pair of images,
c2) determining the difference image by a pixel-by-pixel subtraction of image brightness values of the two images.
In one embodiment the image variation threshold value initially established in step d1) is calculated depending on the image brightness values (actually) occurring in the difference image.
In one embodiment in step d2) the number of image variation points in the form of isolated pixels and/or isolated relatively small cohesive pixel regions in the difference image not exceeding a predetermined maximum number is used as a quality criterion.
In one embodiment one or more criteria used within the scope of the object detection method, in particular, e.g., the quality criterion used in step d2), is/are changeable by a user input.
In one embodiment in step e) the object points are determined respectively as image regions of image variation points lying closely together determined in step d).
According to a still further aspect of the invention a device is provided comprising means for carrying out a method of the type described above in particular comprising an imaging sensor device for generating the video image sequence and a program-controlled computer device (e.g., a processor) for carrying out steps a) through e).
A preferred use of a device of this type or the object detection method carried out therewith can be used for manned or unmanned missiles, in particular for so-called target acquisition or target tracking.